P-289 Evaluation of AI-based, non-invasive and annotation free EMBRYOAID software with embryologists: time and prediction

Abstract Study question Is an AI model as good experienced embryologist? Summary answer Properly trained models can perform embryologists with respect to accuracy, improving in the same time decisiveness. What is known already There are many attempts solve embryo selection problem. One way determine specific features of and on this basis calculate final score. The non-algorithmic approach uses professional knowledge embryologist visual analysis score embryos. most promising using deep learning, specifically CNNs directly predict pregnancy probabilities from a given image or set images. Although these tools deliver high quality answers they have rather low intra-embryologist agreement. design, size, duration Comparing scoring algorithms challenge, miss common scale, e.g., total ranking In order overcome problem we designed test containing150 pairs day-5 time-lapses. For each pair embryos only one gave (implantation based beta-hCG). We compared our algorithm decisions 10 years experience average. Participants/materials, setting, methods created web questionnaire for test. It displayed time-lapses allowed choose more one. invited doctors several clinics take part study. was tested data, i.e., its goal between two transferred After collection effectiveness were compared. Main results role chance comparison follows. accuracy predicting that was: - 66.9 (CI 63.1 70.7) model, 63.8 62.6 65.0) average embryologists. taken by slightly better, however, holds statistical significance. Some variance, there cases where 5out indicated embryo. understand variances divided test-set into parts: a) 57 all agreed decision, b) 93 some differences. On experts 95% cases. While correlation expert ground truth weak,i.e., p-score approximately 0.1. last aspect study making decision. 54 seconds while took 2 Limitations, reasons caution experiment shows agreement case when agree. However, difference scores Wider implications findings returns result almost immediately, thus it speed up process selecting significant agrees Trial registration number not applicable